Exploration by Maximizing Renyi Entropy for Reward-Free RL Framework

Exploration is essential for reinforcement learning (RL). To face the challenges of exploration, we consider a reward-free RL framework that completely separates exploration from exploitation and brings new algorithms. In phase, agent learns an exploratory policy by interacting with environment collects dataset transitions executing policy. planning computes good any reward function based on without further environment. This suitable meta setting where there are many functions interest. propose to maximize Renyi entropy over state-action space justify this objective theoretically. The success using as results its encouragement explore hard-to-reach state-actions. We deduce gradient formulation design practical algorithm can deal complex environments. solve policies given arbitrary batch algorithm. Empirically, show our effective sample efficient, in superior phase.